thanks for fixing it!

BTW is there a way to capture it in PR builders? Otherwise, it's too easy to forget checking...

This seems to disabled by ENABLE_DOCKER_INTEGRATION_TESTS intentionally and designed to be turned on docker module change only at the original commit (2de19e4).

cc @sarutak and @HyukjinKwon .

I'll make a followup PR for SPARK-35483 to enable it for SQL part change, too.

@dongjoon-hyun
I think It's because docker-integration-tests don't depend on sql module in dev/sparktestsupport/modules.py.

We intentionally let the dependency empty after the discussion.
If it's better to depends on sql module, I'll fix it.

Here is the follow-up for SPARK-35483.

• https://github.com/apache/spark/pull/33125/files

Docker integration test passed.

• https://github.com/dongjoon-hyun/spark/runs/2938755266?check_suite_focus=true

Merged to master. Thank you, @cloud-fan , @sarutak , @HyukjinKwon .

Thanks @dongjoon-hyun!